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IDENTIFIERS 
ABSTRACT 

The present study was designed to answer four 
questions:, (1) do anonymity and purpose affect the leniency of self 
ratings in general, (2) do these factors interact, (3) do self . 
ratings approximate supervisor ratings under certain conditions, and 
(4) is self-assessment a useful method of determining training needs? 
Self-ratings and supervisory ratings were made for 206 employees who 
were in one of three job categories in the United States Marshals 
Service. Subjects were told that ratings were for one of three 
purposes: training needs assessment, research, or personnel 
administration. In addition, they were instructed either to sign or 
r> t to sign their names. The results indicated that the stated 
y pose for self ratings had a significant effect on self rating 
leniency, and anonymity had no effect on self rating leniency. 
(BW) 
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Employee self ratings are not widely used in training needs 
assessment, primarily because o.f the frequently cited finding of 
self rating leniency (Thornton, 1980) . However, self ratings have 
several advantages which make them useful for identifying 
training needs. First, they are economical. Second, individuals 
may have access to/ more information about their ' abilities and 
performance than do observers (Jones f, Nisbett, 1971). Third, 
self evaluations typically show less halo than do supervisory 
ratings, which indicates that there is greater discrimination 
between rated factors for self ratings. If self rating leniency 

» 

may be reduced, self ratings may be a usefujl method of 
identifying training needs. 

The purpose for which the self ratings are made and the 
anonymity of the ratings are two factors which may influence self 
riting leniency (Landy ft Farr, 1980; Wabe ft West, 1982). With 

respect to anonymity, Mabe and West (1982) suggested that when 

i 

self ratings are anonymous, the self-esteem of the rater would 
not be enhanced by high ratings. Thus, anonymous ratings should 
be less lenient than named ratings. Regarding purpose, if the 
purpose if such that an individual is likely to benefit from 
distortion, as in a selection or promotion procedure, then 
increased leniency is a potential problem. 

In addition, anonymity and rating purpose may interact. 
When ratings are made for administrative reasons there is a high 
likelihood that spuriously high ratings may be viewed by an 
individual who can invalidate them. In this case, the individual 
may benefit nore from accurate than from inflated ratings. 
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when the stated purpose is research rather than 
administration, the individual making a self-assessment is likely 
to derive . ittle observable benefit from distorted ratings. 
However, ratings which can be -identified may be inflated because 
heightened self-esteem may result for the rater. Even if the 
results can be checked for accuracy, which is an unlikely 
condition if the ratings are to be done for research purposes, 
the individual would not be penalized for inaccurate results. 

Therefore, anonymous ratings made for research may show less 
leniency than named ratings, while confidential administrative 
ratings may exhibit more leniency than identifiable evaluations. 
Such effects may not typically appear since research ratings are 

usually anonymous or confidential while administrative ratings 

I 

can usually be identified. 

The present study was designed to answer four questions. 
First, do anonymity and purpose affect the leniency of self 
ratings in general? Second, do these factors interact as has 
been suggested above? Third, do self ratings approximate 
supervisor ratings under certain conditions? Fourth, is self- 
assessment <i useful method of determining training needs? 

Pethod 

Subje cts : Self -ratings were made by 206 employees who were 
in one of three job categories in the United States Marshals 
Service (tff.KS) . Incumbents in these positions perform a variety 
of federal law enforcement tasks. In addition, each employee's 
immediate supervisor also rated the incumbent. 

fleasu itH : A rating booklet identifying 66 trainable 
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knowledges and .skills which are important in the three jobs of 
interest was prepared* Incumbents 1 proficiency on each knowledge 
and skill was xated using a'9-point scale with anchors ranging 
from "inadegua t* H to "outstanding" • In addition , overall 
performance was rated \ng the same scale. 

Proce du re : Incumbents were asked to complete one 
questionnaire indicating their proficiency on each of the 66 

knowledges and skills and their overall performance level* 

/ 

Subjects were told that ratings were for one of three purposes; 
training needs assessment, research, or personnel^administ ration. 

In addition i they were? instructed either to sign or not to sign 

i 

their names. v 

Incuotbents work in one of 90 districts. Each district was 
randomly assigned to one of the six instruction conditions, and 
all employees in that district received the same set of 
instructions. A questionnaire was mailed to each incumbent with 
instructions frojn the Director of the TISHS to return the booklets 
promptly. .Supervisory guestionnaires were mailed two weeks after 
the rating forms had been mailed to the incumbents. After both 
the self- find supervisory rating booklets had been returned and 
all analyses had been completed, a debriefing letter was sent to 
all employees and supervisors. 

Results 

Two dependent measurer were used for all analyses except 
those for halo. The first measure was the overall performance 
rating, and the second was the mean knowledge and skill rating, 
averaged across the 66 items. 



etform&ncf* measure (F (2,1f>9) - 1.22, i>=.2' 
i/ility measure (F (2, IBB) =0.97, £=.382). 



PAGE U 

. The first nticstion of interest was whether or not anonymity 
and purpose affected self rating level. An analyses of variance 
indicated that, there was a significant effect of rating purpose 
on. leniency for both the performance and the average ability 
ratings (F (2, 169) =4.93, i> *.008; F (2, 188) =4.28, £=.015, 
respectively) . Anonymity did not have a significant effect on 
either the performance measure (F (1, 169) -1 .89, £-.171) or the 
average ability measure {F (1 , 188) -1.04, £-.310) . There were no 
significant anonymity-by-purpose interactions for either the 
performance measure (F (2,1f>9) - 1.22, £>=.298) or the average 
al 

^__Iix-oii]er to determine which group differences were 
significant , Student Newman-Keuls paired comparison tests (Kirk, 
1982) were performed for the purpose variable. For the 
performance ratino, the administrative and research conditions 
did not differ significantly, while the mean of the performance 
ratings in the training group was significantly lower than both 
the administrative and research condition^ (£<.Q5) . No 
significant pairwise differences were obtained for the average 
ability rat ing . 

Although the six-group anonymity-by-purpose interaction was 
not significant, the second guestion. concerned a possible 
interaction between anonymity and purpose for the research and 
administrative group* only. The hypothesized four-group 
interaction was not obtained for the ability measure; thus, no 
comparison was performed for this measure. For the performance 
measure the four-group interaction war; in the predicted 
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direction, planned comparison indicated that the differences 
between the groups were not significant (F (1 , 1b9) -2.81 , £-.10). 

The third question of interest was whether self ratings can 
approximate supervisory ra tings under certain conditions. 
Overall, the dean self rating for both the performance measure 
and the average ability rating were significantly higher than the 
mean supervisory ratings [t± (156) -3.67, £=.0005 for performance; 
LK169) -3.K9, £-.0005 for average ability). The overall 
correlation between the self and supervisor performance ratings 
was only .0U (i>-.60) , and the overall correlation between the 
average ability measures was .12 (£-•12). These correlations did 
not vary wh^n looked at as a function of instructional condition. 

Thn final question concerned the use of self ratings for 
training needs assessment. In addition to the leniency of the 
ratings, the amount of halo present is also relevant. For the 
halo analysis, only items which met two criteria were included. 
First, an item had to be equally important for all three jobs. 
Second, items selected for the halo analysis were chosen to 
represent conceptually different categories of knowledges and 
skills, thus representing separate dimensions of performance. 
One operational definition of halo requires an analysis of the 
item intrrcorrelations . The average z 1 coefficient for the 
supervisory ratings was 0.673, and the average z 1 coefficient for 
the self Tk\ tings was 0.365. This difference was significant, 
(2,-2. P1, £-.003), indicating lower halo for self than for 
supervisory ratings* 

A second operational definition assumes that restricted 
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variability across dimensions is due to a halo effect * The 
supervisor ratings showed a significantly smaller average 
standard deviation across rated abilities than the self ratings 
(t^(169) -2.17, g«.-025) . Thus, ixsing both definitions of halo, 



the self ratincjs showed a significantly smaller halo effect than 
the supervisor ratings. 

Looking at self ratings alone , the correlational definition 
of halo did not indicate differences in the amount of halo 
exhibited across the six treatment groups. Using the standard 
deviation definition of halo, the purpose manipulation had a 
significant effect on self rating halo; a paired comparison 
indicated that the training group had significantly less halo 
than the other two groups (j><.0 r j). / 

The variance of the supervisory ratings was significantly 
greater than the self rating variance for both dependent measures 
(Fornax (?, 169) =3.99, i><.(>1 ^ or th<? performance measure; Fo- 
rnax (2, 16 9} -2 .5H # ]><.01 f° r ***** average ability rating) • 



with respfect to purpose, the results indicated that the 
stated purpose foi self ratings had a significant effect on self 
rating lenienc y for both* dependent measures, when subjects were 
told that Him purpose of th*> ratings was training needs 
assessment, both dependent measures w<^re less inflated than when 
th* stated purpose was research or administration, although this 
difference was on]y significant for the performance measure. 

Anonymity had no effect on self rating leniency for either 
rU'p^mient measure. There are thr^e possible reasons for this 



Conclusions 
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lack of an effect* First, the subjects may not have believed 
that the questionnaires vwe in fact anonymous. A second 
possibility is that due to the nature of their jobs, the 
incumbent* are simply »oi> cautious Mian most populations in 
filling out questionnaires. A third possibility is that the 
performance and ability measures used here are less sensitive to 
anonymity manipulatibns than are the attitude measures for which 
anonymity effects have been reported (cf. Gordon & Petty, 1971) • 
The failure to find the predicted four-group interaction 
indicates that the effect of rating purpose on leniency is not 
dependent upon the identii iability of the ratings. 

The self ratings showed less halo than the supervisor 
ratings using both correlational and inter-item standard 
deviation operational definitions of halo. In addition, using 
the standard deviation operational definition, the training self 
ratings showed the least halo of the three purpose conditions. 

Self-ratings exhibited less variance than supervisor 
ratings. In addition, for both dependent measures, self ratinys 
were found to be more lenient than the corresponding supervisor 
ratings. However, given that the training self ratings were less 
lenient than the research and administrative ratings, and that 
self ratings in general (and training self ratings in particular) 
showed le«;s halo than did the supervisor ratings, self ratings 
may indeed be a useful method of determining training needs. 
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